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Abstract: In the US, many federal, state and local school improvement policies rely on teacher 
professional development (PD) to foster classroom change. Past research suggests PD that has a 
content focus is the most effective, but that even content-focused PD varies in its effectiveness. 
Through in-depth interviews of teachers participating in a middle school science PD randomized 
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control trial in the US, we find that what teachers learn in PD varies significantly based on their 
prior knowledge and experience. This paper explores several hypotheses about how content 
knowledge and teacher learning interact. We conclude that the next step toward improving teacher 
PD is to calibrate learning opportunities to teachers’ prior knowledge. 

Keywords: learning policy; teacher professional development; teacher learning; content knowledge; 
pedagogical content knowledge 

Ideas sobre como dar forma a la politica de aprendizaje de los maestros: El papel del 
conocimiento del contenido en la explicacion de los efectos diferenciales de desarrollo 
profesional 

Resumen: En los Estados Unidos, muchas de las reglas y politicas en las escuelas locales, estatales y 
federales se basan en el desarrollo profesional (DP) para promover cambios positivos en los salones. 
Investigaciones anteriores han sugerido que el DP se ha centrado en el contenido que es el mas 
eficaz, pero incluso existe el DP centrado en un contenido que suele variar en su efectividad. A 
traves de entrevistas con profesores que participaron en un estudio de control aleatorio de DP en 
una escuela secundaria de ciencia en los EE.UU., descubrimos que lo que los maestros aprenden en 
el DP varia significativamente en funcion a sus conocimientos y experiencia previa. Este articulo 
explora algunas hipotesis acerca de como interactuan el conocimiento del contenido y los maestros 
de aprendizaje. Llegamos a la conclusion de que el siguiente paso para mejorar el DP de los maestros 
es crear oportunidades de aprendizaje de conocimiento previo de los profesores. 

Palabras clave: politica de aprendizaje; el desarrollo profesional de los maestros; maestros de 
aprendizaje; el conocimiento del contenido; conocimiento didactico del contenido 

Uma visao sobre como dar forma na politica de aprendizagem de professores: O papel do 
professor em conhecimento de conteudo em explicar efeitos diferenciais de 
desenvolvimento professional 

Resumo: Nos Estados Unidos, muitas politicas de melhorias nas escolas municipals, estaduais e 
federais dependem de desenvolvimento profissional de professores (DP) para promover a mudan^a 
de sala de aula. Pesquisas anteriores sugerem DP que tem um foco de conteudo e o mais eficaz, mas 
que mesmo DP focado em um conteudo varia em sua eficacia. Atraves de entrevistas com 
professores que participant de uma tentativa de controle aleatorio de DP em uma escola de ciencias 
de ensino medio nos EUA, achamos que o que professores aprendem em DP varia 
significativamente com base no seu conhecimento e experiencia previa. Este artigo explora algumas 
hipoteses sobre como o conhecimento de conteudo e aprendizagem de professores interagem. 
Concluimos que o proximo passo para melhorar DP de professores e calibrar oportunidades de 
aprendizagem ao conhecimento previo dos professores. 

Palavras-chave: politica de aprendizagem; desenvolvimento profissional de professores; 
aprendizagem de professores; conhecimento do conteudo; conhecimento pedagogico do conteudo 

Introduction 

Teacher learning is a central component of many school reform policies at the national, 
state, and local levels in the United States. Standards-based reform, comprehensive school reform, 
accountability systems such as those developed under educational policies such as No Child Left 
Behind, and curriculum reforms like the recent Common Core State Standards (CCSS) in the United 
States are no exception. There are already multiple state efforts focused on designing and developing 
professional development (PD) to help teachers build their instructional skills and content 
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knowledge (CK) in order to improve their teaching in ways that respond to the CCSS. For example, 
the role of PD in supporting teachers is especially emphasized in recent science reforms, including A 
Framework forK-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas (National Research 
Council, 2012) and the Next Generation Science Standards (NGSS Lead States, 2013). In other 
words, teacher CK and pedagogical skills play important roles in the success of the implementation 
of many reforms and, as such, we need to have a better understanding of the policies and practices 
related to teacher learning. 

What do we know about building teachers’ CK and instructional skills to improve student 
learning? Previous research supports several general principles that are important for the 
effectiveness of PD. First, building teachers’ CK in meaningful ways requires sustained, content- 
focused PD that is embedded in teachers’ work lives and that allows for practice, discussion, and 
feedback (Ball & Cohen, 1999; Garet, Birman, Porter, Yoon, & Desimone, 2001; Ingvarson, Meiers, 
& Beavis, 2005; Little, 1982, Loucks-Horsley, Hewson, Love, & Stiles, 1998). Second, translating CK 
into improved instructional strategies is not automatic; the process works through various 
mechanisms including increased confidence, willingness to ask higher level questions, proclivity to 
experiment, ability to identify student mistakes, and many others (e.g., Bitan-Friedlander, Dreyfus, & 
Milgrom, 2004; Hill, Rowan, & Ball, 2005; Lee & Luft, 2008; Maskit, 2011; Parker & Heywood, 
2000). Third, teachers vary considerably in what they learn and how they translate that knowledge 
into practice, and we do not know as much as we should about why this variation occurs. That is, 
given a common high-quality PD experience, there is considerable variation in the extent to which 
teachers increase their knowledge, implement new practices with high fidelity, understand the 
changes, and elicit effects on student achievement (e.g., Domitrovich et al., 2009; Garet et al., 2010; 
Gowlett et al., 2015; Penuel, Fishman, Yamaguchi, & Gallagher, 2007). In order to understand how 
to better shape and target PD so that it is effective for all teachers, it is important that we have a 
fuller understanding of this variation in PD effectiveness. 

Our study explores the differential effectiveness of teacher PD through teachers’ reflections 
on their learning. We seek to unpack why so-called “high-quality” PD—PD that is focused on 
content, provides active learning opportunities, is coherent with the teacher’s curriculum and other 
school efforts, is of sufficient duration (e.g., lasts 20 hours or more over a span of several months), 
and provides opportunities for teachers to interact with others (see Desimone, 2009; Yoon et al., 
2007)—has such varying effects on teachers. Although a myriad of factors may explain this 
variation, including context, organization, personal characteristics, beliefs, leadership, and 
curriculum, we focus here on teacher CK. We ground our work in the strong research base linking 
CK to the effectiveness of teachers’ instruction (Cohen, McLaughlin, & Talbert, 1993; Hill, Ball, & 
Schilling, 2008) and the substantial work investigating the interaction of content and pedagogy in 
teacher training (Ball, 2000; Boyd, Grossman, Lankford, Loeb, & Wyckoff, 2009; Darling- 
Hammond, 1997; Grossman, 1990; Putnam & Borko, 1997). 

The analysis presented here focuses on the following broad question related to explaining 
variation in teacher learning and effects from high-quality PD: How does teacher CK moderate the effects of 
PD on teachers, and how does the nature of the PD—whether it is focused on content only or content and pedagogy — 
influence this relationship? We focus on differences between two groups of teachers: those with strong 
middle school science CK, and those with weak middle school science CK. Although we describe 
variation within these two groups, in this study our primary focus is on between-group variation. 
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PD and Teacher Knowledge and Practice 

Central in widely held theories of action for PD is the notion that teachers’ learning 
experiences, in combination with a host of contextual and personal factors, drive changes in what 
teachers know and do (Borko, 2004; Desimone, 2009; Supovitz, 2001). A collective understanding of 
key features of PD that are most likely to lead to changes in teacher knowledge and instructional 
practice has emerged: prioritizing an emphasis on the content of teachers’ instruction, the collective 
participation of teachers from the same school, an extended focus on a particular PD initiative, and 
a close connection to teachers’ day-to-day work of teaching, among other features (Garet et al., 

2001). Translating PD content and implementing new instructional practices so that they can be 
used effectively in the classroom is a complex process. We know that such translation is moderated 
by teachers’ prior knowledge, beliefs, and practices, even if we do not know precisely how. 

Cohen (1990) argued that imperfect implementation of an intervention is related to a lack of 
opportunities for observation and feedback, too little assistance in integrating previous practices 
with new practices, and insufficient understanding of the subject and the goals of the reform. Borko 
and Putnam built on Cohen’s work by emphasizing that PD learning experiences need to help 
teachers build new knowledge and beliefs in order to be effective (Borko & Putnam, 1996). Further, 
Borko and Putnam provided us with an overview of the complexity and importance of social 
interactions in teacher learning (Borko, 2004; Putnam & Borko, 1997). Their work emphasized the 
importance of considering teacher context and classroom experiences in shaping policies and 
practices to promote teacher learning. 

Related research efforts over the past 3 decades have aimed to uncover the particular nature 
of knowledge used in teaching in order to improve teacher effectiveness. These lines of work have 
fostered a general acceptance of the notion that general CK in a subject like science, such as might 
be imparted via a university textbook, is distinct from pedagogical content knowledge (PCK), or the 
specialized knowledge necessary to convey content effectively to students. Furthermore, PCK is 
theorized to be multifaceted, incorporating aspects of knowledge such as knowing appropriate 
models to illustrate concepts and understanding how students learn in a particular content area 
(Shulman, 1986). Work by Ball and her colleagues has focused on further defining the particular 
domains of knowledge for teaching, and these researchers have provided empirical evidence of the 
role of a profound understanding of instructional content in enabling teachers to instmct in ways 
that support student learning (Ball, 1990, 1991; Ball & Rowan, 2004; Ball, Lubienski, & Mewborn, 
2001). Their arguments make clear that this type of knowledge is not simple or straightforward to 
attain. They identified links between CK for teaching and the use of certain instructional strategies 
that elicit student learning, such as identifying student mistakes and engaging in deeper level 
conversations (Ball, Thames, & Phelps, 2008; Hill, Rowan, & Ball, 2005). 

These general principles have been shown to apply across content areas, including science. 
While teacher knowledge in science has not received as much attention as in mathematics (see 
Diamond, Maerten-Rivera, Rohrer, & Lee, 2014), research has established a link between teachers’ 
knowledge of science and their students’ science achievement (e.g., Fleer, 2009; Shallcross, Spink, 
Stephenson, & Warwick, 2002; Supovitz & Turner, 2000). As in mathematics, there seems to be a 
consensus that elementary and middle school teachers on average do not have the strong science 
knowledge needed to foster 21 st -century science learning (e.g., Jiittner, Boone, Park, & Neuhaus, 
2013; Nowicki, Sullivan-Watts, Shim, Young, & Pockalny, 2013), but that when they do, they are 
better able to foster deep and meaningful student learning across multiple science domains 
(Diamond et al., 2014). 
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These ideas stressing the important links among teacher knowledge, instmction, and student 
learning have been the focus of many conceptual and empirical expositions across different subjects, 
including science (e.g.. Diamond et al., 2014; Maerten-Rivera, Ahn, Lanier, Diaz, & Lee, in press; 
Schmidt, McKnight, & Raizen, 1997), mathematics (e.g., Borko et al., 1992), and English 
(Grossman, 1989). 


Toward an Understanding of PD Impacts 

The work of these researchers has guided PD designers and experimentalists in testing PD 
interventions in randomized controlled trials (RCTs) in order to understand how to better develop 
effective PD as part of district and school improvement policies. The links to this foundational work 
are obvious in the consistent focus on content-focused PD with various features that the previous 
work has extolled as “high-quality” and necessary to elicit teacher change, such as practice 
opportunities, links to the classroom, sustained learning experiences, and collaborations with other 
teachers (Borman, Gamoran, & Bowdon, 2008; Garet et al., 2008, 2010; Penuel et al., 2007). 

The effects of these experiments are mixed. Some show positive effects on increased 
knowledge of both teachers and students (Heller, Daeler, Wong, Shinohara, & Miratrix, 2012), 
demonstrating, for example, that PD in science can improve both teacher and student learning 
(Maerten-Rivera, Huggins, & Adamson, 2013). Many have positive effects on teachers but not on 
students (Garet et al., 2010). Some have effects on particular groups of students but fail to show 
average effects (e.g., Borman, Gamoran, & Bowdon, 2008). Still others show no differences among 
treatment and control teachers, even with an intensive high-quality PD experience (e.g., Davidson, 
Fields, & Yang, 2009). Measurement, design, power, and other technical issues aside, how might we 
interpret findings from these rigorous, causal studies that do not demonstrate strong links between 
PD participation and knowledge and practice change that we might expect, given findings from the 
foundational work establishing the PD—knowledge—practice links? A careful examination of recent 
rigorous PD experiments shows repeatedly that the same PD often results in variation in effects on 
teachers and students based on teacher characteristics such as novice status and, most often, CK 
(Davidson et al., 2009; Santagata, Kersting, Giwin, & Stigler, 2012; Yoon, Liu, & Goh, 2010). Thus, 
while considerable evidence suggests variation in teacher response to PD, there is little focus on 
understanding how and why teacher differences result in variation in outcomes (for recent 
consideration of variation of within-school policy take up, see Gowlett et al., 2015). 

A Middle School Science Study Provides an Opportunity to Investigate 

To shed light on this question about the differential effectiveness of PD, we conducted an 
in-depth interview study of a subset of teachers participating in a three-armed RCT focused on 
middle school science teachers and their students. The study provided an opportunity to explore 
teacher experiences in a high-quality PD program and their resulting reflections on their knowledge 
and instmctional growth. We have quantitative assessments of teachers’ CK and classroom 
implementation as well as qualitative interview data on their PD experiences and classroom 
implementation. This mix of quantitative and qualitative data enables us to explore the links among 
the nature of PD, teacher CK, and the perceived effects of PD on teacher learning and instmction. 

We seek to help explain how teacher CK moderates teacher learning and responses to PD as 
a way to gain a better understanding of how to effectively shape and target PD policies and 
programs. We describe trends in the relationship between teacher CK and teachers’ views about 
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what they learn in PD, and how this differs depending on the nature of the PD—whether it focuses 
purely on subject matter content or includes both content and pedagogical strategies. 

Theory of Action 

In Figure 1, we present our theory of action, which we apply here to teacher CK, although it 
can be applied to other teacher characteristics. We recognize that our theory of action includes only 
some of the important and complex relationships that stem from high-quality PD; we do not intend 
the model to be all encompassing. Instead, we use our model to help frame an understanding of 
what emerged from our interviews regarding relationships among prior CK and CK gained through 
the PD, PCK, and affective outcomes (i.e., confidence, risk taking, etc.), each of which have been 
shown to be related to student achievement (e.g., Buczynski & Hansen, 2010; Hill, Rowan, & Ball, 
2005; Ingvarson et al., 2005; Penuel et al., 2007). Additionally, we note that this theory of action can 
be used for various methods of measuring these relationships—including teacher reflections (as 
done here) or quantitative assessments of learning, for example. 

In Figure 1, we represent the CK, PCK, and affective outcomes gained through PD as 
separate boxes. Within each of those boxes there are two arrow lines—a dashed arrow line to 
represent someone with strong prior CK and a solid arrow line to represent someone with weak 
prior CK. The arrow lines are of varying height to represent what is gained from the high-quality 
professional development. 

We plot potential gains (among many) in order to illustrate our conceptualization. The first 
one is a potential gain for a teacher with strong prior CK, represented by the dashed arrow line. 
Because the teacher already has strong CK prior to the PD, it is likely that the teacher will not gain 
or perceive a large amount of additional CK, indicated by a short dashed line; she may, however, 
notice an increase in her level of PCK and/or confidence in teaching. In other words, high-quality 
PD has the potential to provide teachers with a variety of benefits; however, what the teachers learn 
likely depends on their initial knowledge level. 

Another example is a teacher with weak prior CK, represented by the solid arrow lines. This 
teacher has more potential to increase her CK. However, CK is not the only possible benefit from 
the PD. If a teacher has weaknesses in both CK and PCK, then the teacher may notice a little 
learning from both or she may focus her attention on CK and note an increase in CK. In other 
words, if there are multiple knowledge gaps to fill, the teacher could fill one gap or partially fill both 
gaps. We intend the model to reflect that teachers’ change in CK, PCK, and affective characteristics 
depends on interactions between the quality and content of the PD with the teachers’ prior 
knowledge. 
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Perception of affective 
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High-quality professional development 


Weak Prior Content Knowledge 
"^Strong Prior Content Knowledge 


Figure 1. Theory of Action of the Potential Role of Content Knowledge in Differential Learning 
from Professional Development 
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Description of the Intervention Study 

The broader study that served as the backdrop for our investigation was a three-armed RCT. 
It tested two conditions against a control group. The two main hypotheses of the study were (a) 
student achievement in middle school science will increase when teacher content knowledge 
increases, and (b) student achievement in middle school science will increase to a greater extent 
when teachers are taught key cognitive science principles and how to use them (Bransford & 
Schwartz, 1999; Chi, 2005; Hegarty, Kriz, & Cate, 2003 as cited in Merlino et al., 2008). The 
intervention targeted three middle school science units—biological, geological, and physical sciences. 
The intervention lasted for 3 years, occurring in phases. The first year focused only on the biological 
science unit for 7th-grade teachers. In the second year we added the geological science unit and the 
physical science unit for 8th-grade teachers. The third year included only the geological science and 
physical science units. Teachers could be involved in the study for a total of 3 years, teaching each 
unit twice. 

Content-only Intervention 

In this first condition, teachers participated in a high-quality PD that exposed them to deep 
and advanced content related to the topics covered in each unit. The PD was provided by science 
museum professionals, university professors, and researchers. The teachers also received a binder of 
content-specific material related to topics covered in the unit (Massey, Cleland, & Mandel, 2013). 

For the geology unit that is the focus of this study, content covered topics such as rock 
classifications, plate tectonics, and volcanoes. The PD was pure content knowledge; the teachers 
were not given any explicit guidance as to how to incorporate the content knowledge into their 
courses. We refer to teachers who participated in the content-only intervention as being part of the 
content arm of the study. 

Content-plus-pedagogy Intervention 

In the second condition, which we refer to as the cognitive science, or content-plus- 
pedagogy, arm of the study, teachers received PD that taught them the tenets of four cognitive 
science principles—analytic reasoning (contrasting cases), diagrammatic reasoning (visualizations), 
the role of background knowledge (student misconceptions), and understanding and long-term 
retention (spaced testing). They also received explicit instruction and materials on how to integrate 
these principles into their classroom instmction, including a detailed “Cognitive Science Manual” 
(CSM), prepared by our research team, which provided guidelines, lessons, warm-ups, and 
assessments for each of the three intervention units. Because the CSM was designed to be used in 
place of the district’s planning guide for each of the three units, the topics in the CSM aligned with 
the district’s guide but were modified to integrate each of the four cognitive science principles. Each 
of these principles is grounded in the literature and described in the following sections based on the 
research that was conducted for the funding proposal (Spencer, Desimone, & McMaken, 2011). 

Analytic reasoning using contrasting cases. Analytic reasoning using contrasting cases is 
based on the work of Bransford and Schwartz (Bransford & Schwartz, 1999; Schwartz & Bransford, 
1998). Using contrasting cases helps students learn new material and concepts (Bransford & 
Schwartz, 1999; Centner, Loewenstein, & Thompson, 2003; Kurtz, Miao, & Centner, 2001; 

Schwartz & Bransford, 1998). For example, contrasting cases helps the students identify important 
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features, make inferences, explain their own reasoning, understand the underlying concepts, and 
prepare for future learning (Chi, 2000; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, De 
Leeuw, Chiu, & LaVancher, 1994). The intervention materials asked the teachers to use contrasting 
cases at the beginning of the lesson and to have their students read about and contrast two concepts 
(Merlino et al., 2008). In the geology unit, one contrasting case involved geological time represented 
as a table and as a football field. Students were asked to complete a chart describing how each 
representation depicts when Earth was formed and the length of particular eras. After the students 
spent time reasoning about the similarities and differences of the two representations, the teacher 
led a discussion about the two concepts. Overall, the contrasting case activities usually took one to 
two days (Yang, Porter, Massey, & Merlino, 2016). 

Diagrammatic reasoning using visualizations. The visualizations component was based 
on research indicating that middle school students often have trouble decoding graphs, charts, and 
other visualizations (e.g., Hegarty et al., 2003) (Merlino et al., 2008), and it was designed to improve 
student spatial reasoning skills. The intervention provided teachers with strategies for teaching 
students how to understand visualizations, such as teaching students how to explicitly identify key 
ideas represented by a visualization, and it emphasized the importance of decoding visualizations on 
a regular basis rather than waiting for students to ask about them. 

Student background knowledge. Students bring with them a combination of knowledge 
from informal learning and limited formal science education, and often, informal learning leads to 
misconceptions (Chi, 2005; Vosniadou, Skopeliti, & Ikospentaki, 2004). Our decision to include this 
component emanated from research suggesting that in science a set of common misconceptions 
often go uncorrected (Merlino et al., 2008). However, students do not necessarily hold the same 
misconceptions about a concept, making it difficult for teachers to identify when students have 
misconceptions (Minstrell, 2001). The intervention offered teachers strategies for addressing these 
misconceptions directly (Merlino et al., 2008). For example, students may believe that humans and 
dinosaurs lived on Earth at the same time. Using various representations of geological time, such as 
those from the contrasting cases described above, a teacher can help students recognize that the 
timing of dinosaurs did not overlap with humans. 

Long-term retention through spaced testing. Testing students on material at spaced 
intervals helps them retain that information more so than simply reviewing the material, because 
testing the students at spaced intervals requires students to repeatedly retrieve the relevant 
information (Roediger & Karpicke, 2006). The cognitive science intervention provided teachers with 
quizzes for their students and indicated the particular days on which to give the quizzes. Not only 
did these quizzes include the most recently covered content, but they also included content from 
earlier in the unit. For example, the geology unit began by covering the geological timescale, and 
throughout the unit, the quizzes asked questions about the geological timescale, requiring students 
to retrieve that information from earlier classes and quizzes. 

The Control Group 

Teachers in the control condition engaged in business as usual—no alternative PD was 
offered to those teachers at the time of the study. However, as an incentive for participation, we 
offered to provide the intervention materials and accompanying PD at the end of the study to all 
teachers in the district who wanted them. 
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Structure of the PD 

To reduce confounding, we designed both the content-only and the cognitive science PD to 
have the same structure, duration, and core features despite the content arm’s focus on content only 
and the cognitive science arm’s focus on pedagogy and curricular materials. For both arms, we 
provided a 2.5-day (18 hour) summer PD and four 2-hour professional learning community (PLC) 
sessions during the school year for 2 years, for a total of 34 hours of PD over 6 months for each 
unit. 1 The summer PD in both conditions included a mix of lectures, laboratories, interactive 
discussions, and observations with feedback. The PLC groups convened to preview upcoming 
topics to be covered in the classroom, to discuss teacher experiences with implementing the units, to 
discuss successes, and to problem solving failures. The groups were led by a study facilitator with 
expertise in either content or the cognitive science principles (Massey, Cleland, & Mandel, 2013). 

Outcome Measures 

For the randomized control trial, we used state assessment scores and an end-of-unit test we 
developed and aligned to the content of the intervention PD for each unit. Previous research has 
shown that standardized assessments can be less sensitive to the results of an intervention (May, 
Perez-Johnson, Flaimson, Sattar, & Gleason, 2009; Olsen, Unlu, Jaciw, & Price, 2011; Somers, Zhu, 
& Wong, 2011), so we wanted to also have a more proximal measure of student achievement. Each 
test comprised 18 items drawn from a pool of publicly released items from state tests (from 
nonstudy states), the National Assessment of Educational Progress, and the Trends in International 
Mathematics and Science Study. We used a validated metric to align questions to the content of the 
unmodified curriculum, so as not to bias the assessments in favor of the intervention arm teachers, 
who used the modified curriculum. 

Findings from the Experimental Study 

Findings from the larger study (see Scull, Porter, Massey, & Merlino, 2016; Yang, Porter, 
Massey, & Merlino, 2016), reported in (Yang, Porter, Massey, & Merlino, 2016) show that students 
of teachers in the cognitive science arm of the study showed more improvement in science 
knowledge, compared with students of teachers in the content or control arms, although usually the 
difference was not statistically significant. The most consistent significant result was when 
comparing the cognitive science arm with the content arm on a state standardized test. Flowever, 
there were not significant differences when examining the specifically constmcted test that was 
aligned to the PD content (Yang, Porter, Massey, & Merlino, 2016). These findings provide the 
foundation for exploring one set of relationships that may help explain the differential effects: the 
relationship between teachers’ content knowledge and what they learn in PD. 


1 Teachers who teach multiple units included in the intervention could have up to three sets of the 34-hour 
PD sessions. 
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Methods 

Sample 

Table 1 


Characteristics of Interviewees 






Cognitive 






science 






principles 

Teacher 





implementation 

content 



Years of 

Undergraduate 

score for 

knowledge 

Name 

Arm 

experience 

major 

geology unit 

scores 

Molly 

Cognitive 

science 

3 

Geology 

2.90 

16 

Emily 

Cognitive 

8 

Education 

2.62 

19 

science 





Monica 

Cognitive 

28 

Education 

2.84 

19 


science 





Jasmine 

Cognitive 

science 

2 

Biology 

3.21 

20 

Andrew 

Cognitive 

science 

3 

Geology 

3.13 

22 

Mike 

Cognitive 

2 

Social science 

2.78 b 

22 


science 





Clare 

Content 

23 

Education 

2.29 

21 

Barry 

Content 

3 

Science 

— 

21 

Bob 

Content 

8 

Business 

— 

22 

Malik 

Content 

7 

Social science 

2.58 

17 

Angela 

Control 

10 

Education 

2.52 

19 

Lisa 

Control 

9 

Design 

2.81 

19 

Lloyd 

Control 

15 

Education 

2.42 

11 

Harry 

Control 

6 

Education 

2.67 

14 


Notes: Cognitive science principles implementation score for geology for full sample (Mean=2.69; SD— 0.30; 
Range=2.13-3.56) a and teacher content knowledge scores for full sample (Mean=19.12; SD= 3.08; Range=8- 
25); ‘The focus of this measure was how teachers in the cognitive science arm implement cognitive science 
principles; therefore we do not expect teachers in the content arm to have higher implementation scores than 
control teachers. b We do not have data from all of the teachers for their implementation scores from the 
second year of our study. For Mike, we include his first year implementation score. 

The larger intervention study included teachers in two large cities in the southwestern United 
States and two large cities in the eastern United States. We selected one of the eastern cities (N=87) 
to conduct our in-depth interview study. We focused our investigation on a single unit in order to 
probe more deeply into content knowledge and implementation issues. We chose to focus on 
geology, which is an earth history unit, because it was taught in the winter, when it was less likely to 
be intermpted by standardized testing. 

We selected 14 teachers from the three arms of the study to reflect on their PD experiences 
and resulting changes in their instruction. We chose six from the cognitive science arm 
(overrepresented because the focus of the larger study was on use of cognitive science principles), 
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four from the content arm, and four from the control group. Our goal in selecting teachers was to 
have teachers with the largest exposure to the intervention because these teachers would be more 
likely to show effects on knowledge and instruction (e.g., Yoon, Duncan, Wen-Yu Lee, Scarloss, & 
Shapley, 2007). We selected teachers who were in their second year of participating in the study, 
because research finds that one year of intervention use may not be enough to learn how to use the 
intervention materials (Borman et al., 2008; Fullan, 1991). Also, we selected cognitive science 
teachers with the highest number of PD hours for the unit and who taught multiple sections of 
science, because they had the largest dose of the cognitive science intervention and more 
opportunities for implementation. Finally, we included teachers from the control arm in order to 
compare varying levels of content and content-plus-pedagogy PD delivered by researchers within 
our study to PD delivered outside of the context of our study. The characteristics of the teachers 
who participated in our interviews are provided in Table 1. We recognize that teachers may also vary 
in ways not reflected in our measures. Flowever, given that this is an exploratory study, we are 
seeking patterns of relationships that can be tested with future research. 

Data 


Several data sources informed this study. Quantitative data were collected from a detailed 
teacher implementation survey, as well as a researcher-created CK test given to teachers in the 
cognitive science, content, and control arms of the study. The implementation scores we use in this 
study are a measure of the frequency with which the teachers follow the cognitive science principles. 
A higher score indicates that the teacher has higher fidelity of implementation. While content and 
control teachers did not receive explicit instmction on the cognitive science principles, we measure 
their frequency of use of these principles as well. Qualitative data come from teacher interviews, 
which were recorded and transcribed. We conducted interviews in the winter of the 2011-2012 
school year. We used the larger study’s quantitative database to identify the teachers who fit our 
criteria. Sorting the teachers with the greatest number of PD hours, we contacted individual teachers 
to ask for their participation. For the control teachers, we selected teachers who would be teaching 
the unit for the second time to multiple classes. Each interview was conducted at the teachers’ 
schools. The interviews were scheduled for 2 hours and in practice ranged from 45 minutes to 2 
hours. For each interview, we used a detailed interview protocol that covered teachers’ perceptions 
of the goals of the PD, a description of how the teacher taught a particular lesson, and facilitators of 
and barriers to science instruction. Since the control teachers did not receive PD as part of the 
study, we asked about any PD that they experienced. To maintain confidentiality, we used 
pseudonyms for the teachers. 

In analyzing the interview data, we followed the procedures outlined by Miles and 
Fluberman (1994), Fluberman and Miles (1994), Patton (1990), and Coffey and Atkinson (1996). 

Our conceptual framework and research questions served as the basis for our initial coding 
framework for interview transcripts (Alexander, 2001). We then added more themes and subthemes 
as called for by our ongoing analysis of the transcript data. We used the constant comparative 
method to develop the codes (Glaser & Strauss, 1967; Strauss & Corbin, 1998) so that ideas from 
the transcripts were used to expand and refine the coding system. Through this iterative process, we 
changed, adapted, and integrated categories or themes (Goetz & LeCompte, 1984). In this way, we 
were able to interactively identify themes using both our conceptual framework as well as the 
transcript data. This method enabled us to use the data to inductively test our hypotheses, as well as 
to deductively allow other themes and explanations to emerge that we had not anticipated in our 
conceptual framework (Emerson, Fretz, & Shaw, 1995; Green, Dixon, & Zaharlock, 2002). The 
codes we used were derived from our framework and the literature and included, but were not 
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limited to, the following categories: perception of content knowledge in science; experience with 
science teaching; confidence in science; description of learning in the PD and changes in instruction; 
understanding of the PD goals; understanding of the cognitive science principles; self-assessment of 
student science learning, engagement, and motivation; relevant school or classroom 
context/conditions; interacting with other teachers; and teacher autonomy. We illustrate each key 
theme with key quotes as exemplars (see Atkinson, Coffey, & Delamont, 2003), a technique that, 
according to Ryan and Bernard (2003), is “a widely used method for describing themes . . . that lead 
the reader to understand quickly what it may have taken the researcher months or years to figure 
out” (p. 282). Given the modest size of our sample, we thought it pmdent to try to reflect the 
continuum of teacher experiences rather than focus only on those issues that “most” teachers 
discussed. Overall, however, most key themes were reflected in the majority of teacher interviews. 

Measuring Teacher Content Knowledge 

In addition to influencing student learning, prior knowledge also plays a critical role in 
teacher learning. Understanding and assessing teacher CK has been the focus of considerable 
research (Ball, 1990, 1991; Ball & Rowan, 2004; Ball, Lubienski, & Mewborn, 2001). Teacher CK has 
been operationalized in multiple ways, including college major, course taking, years of experience, 
certification, and test scores (Sadler, Sonnert, Coyle, Cook-Smith, & Miller, 2013). While evidence is 
mixed in relating these variables to instructional quality and student learning (e.g., Wilson, Floden, & 
Ferrini-Mundy, 2002), college major has been shown to be related to student achievement 
(Goldhaber & Brewer, 1997, 2000; Rowan, Chiang, & Miller, 1997). Our choice to use college major 
as the measure of teachers’ CK is guided by this general finding that using major as a proxy for 
teacher CK is consistently correlated with expected teacher and student outcomes. Perhaps more 
importantly, teachers in our study who had an undergraduate major in science perceived themselves as 
much stronger in CK than teachers without such a degree. We found this a compelling factor in our 
decision about how to classify teachers’ CK, given that our investigation is primarily concerned with 
teachers’ views of their own CK and how they believe their CK influenced what they learned from 
the PD. 

This is not to discount the considerable CK growth that can occur through other 
mechanisms, such as years of teaching particular subject matter to students, participating in in- 
service learning opportunities, self-guided subject-matter reading, and many more. Investigating the 
relationship between multiple measures of teachers’ CK is beyond the scope of this study. That 
being said, we do note that teacher scores on the CK test we created and administered as part of the 
study were generally, though very weakly, related to undergraduate major. For the teachers we 
studied, the average CK score was 18.7 (range 8-25) (see Table 1), with little difference between 
those with and without a science major—on average, those with a science major scored a 19.75 and 
those with a non-science major scored 18.30. While the CK test we created was reliable and valid 
(Porter, Polikoff, Barghaus, & Yang, 2013), it measures a very narrow domain of middle school 
science (as is the nature of most tests) and as such was unlikely to capture much of the differences in 
what teachers know about middle school science and how to teach it. 

We recognize that teacher CK is complex, and is not fully represented by undergraduate 
major, a knowledge test, or any other single measure. We feel comfortable using undergraduate 
major as an indication of CK here given the exploratory nature of our study, and given the 
correlation between undergraduate major and teachers’ description of their confidence and comfort 
level with teaching science. 



Education Volicy Analysis Archives Vol. 24 No. 61 


14 


Results 

Our analysis uncovered several themes regarding teachers’ learning in the content-only and 
cognitive science interventions. We describe these themes below as lessons, each contributing to a 
deeper understanding of the role of CK in the effects of PD on teachers. 

Lesson 1: What Teachers Learn and Do as a Result of High-Quality Content PD Depends 
on Their Prior Knowledge 

We find that when teachers participated in the high-quality content-focused PD offered in 
our study, what they perceived that they learned and did as a result depended on their prior CK. 
Through our analysis of the teachers in the content-only arm, we found that teachers with weak 
prior CK (i.e., a nonscience major) made gains on multiple dimensions: CK, PCK, and affective 
factors such as confidence. However, if teachers already had substantial CK in the content area 
covered, they tended to focus on increasing only their PCK. 

Clare, who was in the content-only arm and did not have a science background, found that 
the content PD helped her broaden and deepen her CK. Clare was a 23-year teaching veteran who, 
by her own admission, struggled with advanced science concepts: 

[Before the PLCs] I basically taught sticking to the facts that come from the 
book, but after the PLCs and after the professional development they pointed 
out how the book has some misconceptions, so that was an eye-opener for 
me because ... there was some other things that I learned that I had been 
teaching and it was wrong because of the information that I was getting from 
the text. 

Clare felt that the content intervention PD helped her build her knowledge of science, and she 
indicated that as a result of her increased CK, her confidence significantly increased and empowered 
her to deviate from the curriculum. She described how her limited knowledge of science content 
made it difficult to answer questions from more advanced students and stifled classroom 
conversations. However, the additional CK learned in the PD helped her to answer those questions, 
and participation in the PLCs gave her “a little bit more heart” to venture away from the district 
pacing guide. Since the PD, Clare applied her content PD experiences to her classroom in such a 
way that empowered her to take instructional risks. She stated that her increased confidence in 
teaching made her more comfortable when engaging with her more advanced students. She was 
better able to support risk-taking in her class, which significantly increased student curiosity. 

Like Clare, Lisa, a control teacher, also experienced high-quality content PD through taking 
a physics course at a local university. While the PD of the course was not specific to geology (our 
discipline of interest), her response as a control teacher, who did not receive any specific PD from 
us, illustrates that high-quality content PD can help improve understanding and confidence. When 
probed to describe the ways the PD broadened her understanding, Lisa stated that the physics 
course changed her instmction and her confidence in teaching. She said. 

The physics course made me understand.. .chemistry in a whole new way and that 
happened at the very beginning when .. .we talked about energy and the energy of 
elements and the energy of everything, of matter period. And so when I went to go 
teach my kids about what’s the difference between a gas, a liquid and a solid, I taught 
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it to them in terms of, who’s got the most energy? Like who’s moving the fastest 
and, and what are they doing with that? And so it, it broadened my horizons you 
know in ... my understanding and the way that I approach teaching it to the kids 
changed. 

When asked to describe how the PD changed her knowledge of science, Lisa went on to say that she 
felt more comfortable having classroom discussions with her students because she had more 
confidence in her knowledge and an increased willingness to admit that she did not know the answer 
to every question. 

Clare’s and Lisa’s learning trajectories resemble the solid arrow lines in Figure 1. Both came 
to high-quality content-focused PD experiences with weak prior CK, and through their experiences 
they believe that they increased their CK as well as their confidence in their teaching abilities. The 
experiences of Clare and Lisa are precisely what previous research would predict, in that building 
teachers’ CK enables them to probe more deeply and to feel more comfortable with material, which 
in turn leads teachers to engage in more exploratory discussions with students, to experiment more 
with ideas, and to rely less heavily on the text (e.g., Cohen et al., 1993). The PD provided them with 
the opportunity to fill in gaps in their CK and build their confidence. 

Filling gaps in CK was not the only response for teachers participating in high-quality 
content-focused PD. Although the content-only arm did not explicitly provide lessons or strategies, 
some teachers still indicated that the main takeaways were related to new instructional activities 
rather than building their CK. For example, Malik, a teacher with a social science background and 
weak science CK, when asked to what extent the PD changed his pedagogy, indicated that the PD 
gave him new ideas and expanded his knowledge base, which gave him more confidence. Although 
he told us that his knowledge base increased, he talked more about how the PD helped increase his 
knowledge of how to do activities in his classroom. He said, 

I mean there was a lot of things obviously I didn’t know because ... it was like my 
second... certification and ... it helped me to ... understand things better and ... 
you know thinking about ... how Pm gonna do different activities and gave me like 
more of a strong knowledge of, of the subject area.... I use like a lot of the, like the 
activities that, that we learned the material from and you know even like ways I look 
at rocks, look at you know the different specimens and how to like describe all those 
things, like what we had to do in, in the class. Then ... I would turn around and I 
would use like a lot of that stuff, you know a lot of that knowledge in the classroom. 

Despite claims of increased content knowledge, Malik primarily talked about ideas for activities 
rather than the CK that he gained. Based on Malik’s description of his learning, it does not appear 
that he increased his CK despite his claims that he did. Instead, descriptions of his activity usage 
indicate that he increased his PCK as well as his confidence and comfort with teaching science. 

The cases of Clare, Lisa, and Malik show that teachers with weak CK can have a variety of 
learning outcomes from high-quality PD even if it is focused specifically on CK. But when teachers 
already have strong prior CK, we found that what they said they learned from PD differed 
substantially from their colleagues with weak CK. 

Unlike other content-arm teachers Malik and Clare, Barry had mastered middle school 
science content. He had a master’s degree in geological science and had worked as a geologist. 
Despite the content PD being devoid of explicit attention to pedagogy, during the PD sessions 
Barry focused on how the facilitators were presenting the material to the teachers. He said, 
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I spent my time because I felt comfortable in the content, I knew I wasn’t 
gonna get, oh this website has this, this website is, you can get this handout 
or this flyer from this location or that location, I spent my time taking notes 
in the margin trying to say, oh this was really good, find out how or where 
this came from [in terms of delivery of the content]... so in that aspect it 
helped me significantly. 

In this case, since Barry already had mastered the CK that was the focus of the PD, he found a way 
to make the PD useful—he focused on aspects related to how the presenters conveyed information. 

These cases highlight a continuing challenge for those who design and study PD: the relative 
emphasis that should be placed on pure CK, on one hand, and instructional or pedagogical 
strategies, on the other. One hypothesis is that building teachers’ CK is enough and it will translate 
into better instruction through multiple mechanisms, such as asking deeper questions and 
understanding student mistakes (Hill et al., 2008). Another hypothesis is that while certain high- 
performing teachers may be able to take increased CK and translate it into better instruction, more 
consistent and stronger effects are found if the PD includes explicit links to lessons, activities, and 
strategies that the teachers can readily translate and integrate into their classroom instruction (Penuel 
et al., 2007). Turning now to the arm of the study that targeted the application of cognitive science 
principles as a way to provide teachers with both new knowledge and new pedagogical strategies 
provides insights into the apparent tension between content-only PD and content-plus-pedagogy 
PD. 

Lesson 2: What Teachers Learn and Do as a Result of High-Quality Content-Plus-Pedagogy 
PD Depends on Their Prior Knowledge 

Our cognitive science arm teachers were split evenly between strong and weak prior CK, as 
indicated by their undergraduate major (as shown in Table 1). As we analyzed the data, we found 
that these two groupings aligned with (a) the extent to which the teachers understood the underlying 
cognitive science theory and (b) based on our assessment of their reflections on teaching and a 
measure of implementation from teacher surveys, the fidelity with which they implemented the 
cognitive science intervention. Teachers who already had the CK were able to describe a deeper 
understanding and higher level of implementation than other teachers. Based on quantitative 
measures of implementation of the cognitive science principles from teacher surveys (see Table 1) 
and qualitative evaluations of implementation from teacher interviews, the three teachers with the 
highest level of implementation were Jasmine, Andrew, and Molly, all three of whom have 
undergraduate degrees in science. The teachers who did not describe a deep understanding of the 
cognitive science principles and had lower levels of implementation—Monica, Mike, and Emily— 
had an elementary education or social science background. 

Jasmine’s, Andrew’s, and Molly’s self-reported learning and change as a result of the 
cognitive science PD resemble the dashed arrow line in Figure 1. All three had strong science 
backgrounds. They saw little gain in CK, but they did find that they gained in their PCK; these gains 
were in the ways that the PD designers had intended. In contrast, the teachers with lower levels of 
prior CK tended not to describe increases in their PCK, but instead reported making gains in CK 
and/or affective outcomes such as confidence and risk taking. We found evidence of the association 
between teacher CK and understanding or implementation in three areas: the level of understanding 
of the cognitive science principles, the extent to which teachers viewed the intervention as a script, 
and the extent to which they transferred ideas from the target unit to other subjects and classes. 
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Understanding the cognitive science principles. Cohen (1990) painted a powerful 
picture of a teacher making an effort to implement a new teaching strategy without mastery of the 
underlying principles. He described how “Mrs. Oublier” introduced mathematical manipulatives to 
her students; she did not seem to know their purpose or how to use them. She asked the children to 
touch them as if they had “magical instructional powers” (Cohen, 1990, p. 318). This demonstrated 
that Mrs. Oublier was following the intervention’s procedures to have the children use the 
manipulatives but did not understand how the manipulatives aided student learning. 

Our findings are analogous to Cohen’s, in that there was variation in the extent to 
which teachers viewed our intervention as a set of strategies to implement in the classroom 
without demonstrating understanding of the underlying principles behind the strategies; and 
we found a positive relationship between teachers’ depth of understanding of the principles 
and those teachers’ CK. 

Andrew, a third-year science teacher with an undergraduate major in geology, 
described the PD as providing theory and research on the principles underlying the use of 
the cognitive science principles. Andrew recognized that the PD provided the teachers with 
more than a binder of lesson plans for the geology unit; however, he also recognized that 
teachers need to implement the activities correctly. He described the goals of the PD as 
follows: 


[The ljarger scale [goals] I guess [were about] helping us as the teachers use the 
materials to the best of our abilities. I mean the work of getting the materials created 
and understanding the principles and the study involved of whether or not are these 
gonna work the best? Is this a different way of teaching that will allow the students 
to understand the concepts better, but the goal, you know these are worthless unless 
the teacher can actually use them effectively. 

Molly, who like Andrew was a third-year science teacher with a geology undergraduate major, also 
saw the PD as a source of information about the underlying theory of cognitive science. When asked 
about the goals of the PD sessions, she said, “I think the goals were just to get us to understand why 
we were doing it in terms of how students learn and the cognitive development that they have.” 

Both Andrew and Molly articulated the underlying theory throughout their interviews. For instance, 
in Molly’s interview she focused on how the PD helped her see how students understand 
relationships among scientific concepts. As a result, she said that her instruction changed to focus 
on the big ideas within a unit rather than on the individual facts, which was a goal of the intervention 
designers. This change in instruction indicates that she internalized the underlying cognitive science 
theory behind the intervention. 

In contrast, the other teachers whom we interviewed in the cognitive science arm of the 
study viewed the PD primarily as a source of materials for their classroom. For example, Emily, an 
eighth-year science teacher with an undergraduate major in elementary education, told us that the 
PD provided her with the chance to learn how to conduct activities with her students. She went on 
to say, 


The one thing that we did that was really helpful because it was kind of a confusing 
activity was the Contrasting Case with the time lines.... And that was helpful, very 
helpful because when you first look at those, you’re like, oh are our kids really gonna 
understand that, but then they ... by doing it with us they were able to say ... you 
know this is what they’re gonna see and this is what they’re supposed to get out of it. 
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In contrast to Andrew and Molly, Emily did not talk about gaining knowledge about the underlying 
principles of the cognitive science intervention. Instead, her concern and focus were on being able 
to repeat the activities on her own. 

Another clear contrast between the teachers who understood the underlying concepts of the 
intervention and those teachers who did not emerged from teachers’ responses when asked why they 
used warm-ups, a component of the CSM, in their classrooms. Once again, there emerged a group 
that understood the underlying concepts and those that did not, which aligned with whether or not 
the teacher had strong prior CK. 

Andrew viewed warm-ups as a method for assessing student understanding, which is the 
purpose intended by the intervention. In contrast, Monica used warm-ups as a behavioral tool rather 
than an assessment tool, saying, 

I just think it’s a good way of just setting the tone, bringing them in, you 
know they’re, they’re coming from upstairs on the third floor, they’ve had 3 
floors, 66 steps to chittychat and when you just go, okay let’s get right into it, 
okay yesterday and they hear that word yesterday they get quiet and boom 
we’re right into it. 

These examples of how teachers described their understanding and used the cognitive science 
principles suggest that the teachers with strong prior CK were able to increase their PCK. They 
tended to understand the main goals of the intervention and the modifications. 

Is it a script or a set of principles? We found that teachers with stronger CK tended to 
describe their use of the CSM as a resource, while teachers with weaker CK were more likely to 
describe their use of the CSM as a scripted lesson plan. Although the CSM is not explicitly scripted, 
some teachers viewed it as a script, due in part to the fact that it provided day-by-day lessons. 

Most of the teachers in the cognitive science arm of the study described the CSM 
modifications as a set of lessons and activities that saved them valuable planning time, and they 
embraced the CSM. However, the level of adherence to the CSM varied based on teacher comfort 
with its content. Emily, who had little science background, strictly followed the CSM. Prior to 
participation in the study, Emily strictly followed the district’s curriculum plan. She indicated that 
the CSM replaced the district curriculum as her instructional script, saying, “[Using the CSM ] 
changed [determining what is important from a lesson] simply because I follow what they do instead 
of following what the school district wants me to do.” Emily relied on the material that she was 
given, either by the intervention or the district, to guide her instruction. Given that Emily tends to 
simply follow the materials that she is given rather than engaging with the material, we argue that she 
did not implement the intervention in a deep and meaningful way. 

Similarly, Monica, who has a degree in elementary education, viewed the CSM as a step-by- 
step guide to instmction. She said. 

That is in ... the notebook, so therefore that works really well that if it says, at this 
point you need to remind students about, or at this point you need to clarify a 
misconception that students might have about. So ... it’s bringing you back, it’s 
grounding you, it’s going, oh yeah, that’s right, I need to do that. 

Rather than internalizing the cognitive science principles, such as being able to anticipate when 
students will have misconceptions, Monica describes the intervention as providing her with materials 
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that explicitly told her the steps that she needed to take. Monica’s responses indicated that she went 
through the motions of the intervention without fully understanding why she did each step. 

Unlike Emily and Monica, Molly and Andrew used the intervention materials as guides to 
lessons rather than as scripts. Molly supplemented the CSM when she felt it was appropriate by 
adding labs and other activities to increase student engagement. And although Andrew did not think 
of the intervention materials as a script to be strictly followed, he did rely on the CSM as a time¬ 
saving source of lesson planning. He said, 

I mean I would, would’ve had to like make up my own ways of which to teach tilings 
and my own ideas about how to incorporate those things on the sidebar into the 
lessons, the actual day to day lessons of what, how that material’s gonna be taught.. .1 
mean this, this is like you know thousands and thousands of hours of, of highly 
educated people coming up with these, these modifications, so I’d rather stick to the 
script, not that this is scripted, but stick to the ideas and principles behind teaching 
that are developed through this program. 

Andrew recognized that the intervention focused on the cognitive science ideas and principles 
behind the activities; however, he also took advantage of the lesson plans and stmcture provided by 
the CSM and focused his energy on the additions that he could make to the geology unit. 

When we examined whether or not teachers viewed the materials as a script, we found that 
the teachers with weak prior CK tend to focus on the activities and the CSM as a script. These 
teachers described going through the motions of the activities, but we have little evidence that they 
truly implemented the principles behind the activities. On the other hand, the teachers with strong 
prior CK used the materials as a starting point, supplementing the material with other activities, 
rather than viewing them as scripts to which they must adhere. 

From principles to practice. One goal of the cognitive science intervention was for 
students to transfer the underlying cognitive science principles they learned to other topics and 
content areas. Although this particular intervention was geared toward middle school science, the 
underlying principles are not content specific. The same goal of transfer can be applied to teachers. 

If teachers truly understood and implemented the cognitive science principles, they should have 
been able to engage in pedagogical practices that reflect these principles with other instructional 
content outside of the focal geology unit. Once again, we saw differences in transfer based on 
teacher CK. 

Monica, one of the teachers with low prior CK, highly praised the cognitive science 
intervention. However, she did not believe that she could create similar lessons for units other than 
those in the intervention, suggesting a lack of understanding of the cognitive science principles. 
When asked if she tried to use the cognitive science materials in other courses, she responded, “Well 
it’s hard because you wouldn’t have your day 1, day 2, you don’t have a CD to go with it, that’s 
where your movies come in you know?” This statement suggests that Monica did not view the 
intervention as an underlying theory for how to instruct students but instead saw it as a source of pre¬ 
made lesson plans. 

In contrast to Monica, Jasmine, a second-year science teacher with an undergraduate biology 
major, tried to use cognitive science principles for units for which she did not have the CSM. She 
said, “You know in the other units that I don’t have the [CSM], I’ve tried, I mean I don’t make up 
Contrasting Cases, but I do have, do a lot of comparing and contrasting. So I try to incorporate 
some of the things that are in the [CSM] into my other, other units.” Using the cognitive science 
principles in other units suggests that the teacher regards the intervention as more than a source of 



Education Volicy Analysis Archives Vol. 24 No. 61 


20 


lesson plans or a series of activities to cover with students. By attempting to use the cognitive 
science approach in nonintervention units, teachers demonstrated that they recognized that the 
principles can be applied to any material. 

We saw clear patterns of teachers with strong prior CK increasing their level of PCK. At the 
same time, we saw teachers with weak CK making superficial changes in their pedagogical practices. 
However, this does not mean that the cognitive science PD was not successful in providing teachers 
with new knowledge. Because the teachers with strong prior CK did not necessarily need to focus 
their attention on the content that was being presented, they were able to make gains in pedagogical 
skills. The teachers with weak prior CK, on the other hand, tended to focus their attention on the 
actual content being presented through the activities and modifications. 

Lesson 3: Teacher Emphasis on Gaining CK versus PCK Depends on Prior Knowledge 

As we have indicated, the cognitive science PD was designed to provide teachers with an 
opportunity to learn how to use cognitive science principles in their teaching, and the PD provided 
them with detailed lessons and activities for how to do that for a particular science unit. The PD was 
not designed to increase teachers’ CK in middle school science—it presented no new content other 
than what was covered in the district’s pacing guide. Although we certainly found that some teachers 
in the cognitive science arm did understand and use the pedagogical strategies in their classrooms, 
we also found that other teachers in the cognitive science arm reported gaining more in content 
knowledge than in pedagogical strategies. 

Teachers with science degrees in our cognitive science arm indicated that the PD did not 
substantially increase their CK, with the exception of a very few specific information points. In 
contrast, teachers without a science background who participated in the cognitive science PD 
reported building their CK. We found that teachers focused their own learning to be consistent with 
their level of science content mastery; those with weak CK focused on the content, while those with 
science backgrounds moved beyond the content to learn the principles and applications. 

Emily, a teacher without a science background, is a good example of this phenomenon. Prior 
to participating in the intervention PD, Emily reported using the textbook as her source of 
information about science; she said that she learned the content along with her students. She was 
unfamiliar with much of the geology unit material, and she admitted to having misconceptions about 
the content. She often did not know the answers to student questions and had to look up the 
answers. In discussing her experiences in the cognitive science PD, she said that the intervention 
helped her gain a better understanding of the geology unit content. She went on to say, “Like I feel 
like I can stand in front of the class and not have to look at my book to say, you know this is this, 
this is this, I can just say to them, like I’m teaching them instead of reading it.” 

As we discussed above, Emily implemented the modifications by following the CSM as a 
script, and she did not demonstrate a deep understanding of the intervention itself beyond its use as 
a pacing guide. We hypothesized that this was because her CK was so weak that her main focus 
during the PD was on gathering the CK that she lacked rather than internalizing the pedagogical 
concepts that were presented in the cognitive science PD. 

Discussion 

Providing effective PD is central to the success of many school improvement policies. In 
this study we explored how teacher CK may explain variation in what teachers learn and do as a 
result of PD, and how teacher knowledge and features of the PD may interact to explain differential 
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effects of PD on teachers. Results should be interpreted in light of the several limitations. Our study 
is exploratory, and thus patterns we see need to be examined on a larger scale. Further, our interview 
and survey data are self-reported, which has both strengths and weaknesses (Mayer, 1999). 

Despite these limitations, we believe our results provide important insights for developing 
PD policy. We found that, with both the content-only and the content-plus-pedagogy interventions, 
teachers used PD to fit their needs, depending on their previous CK in the subject (which in this 
case is science). Even though not all teachers learned what the PD designers intended for them to 
learn, they did report learning something that could help them improve as teachers. 

Teachers in the cognitive science arm who came to their PD with strong CK were much 
more likely to describe a deep and meaningful understanding of the cognitive science principles, and 
to describe how they integrated these principles into their teaching in meaningful ways. In contrast, 
teachers with weak science CK used the PD as a way to bolster their CK, and used the cognitive 
science intervention as more of a script to follow. Returning to Figure 1, the teachers in the 
cognitive science arm with the strong CK background generally followed the dashed line arrow, 
noting gains in their PCK. On the other hand, teachers in the cognitive science arm with weak CK 
generally followed the solid path, describing more of an increase in CK than in PCK when reflecting 
on their growth. 

We found a similar pattern of differentiation in the content-only PD. There, teachers with 
strong CK focused on learning from the PD delivery activities as new ways to present material. 
Across both the content-only and cognitive science arms, we also found examples of teachers with 
weak prior CK gaining knowledge from the PD in ways that seemed to translate to better 
instruction, as previous research would suggest. These teachers felt that their confidence had 
increased and that they were better able to answer their students’ questions, making them better 
science teachers. 


Policy Implications: Differentiating PD 

In the context of the broader literature that emphasizes the importance of teacher quality, 
and the critical role that PD plays in supporting teachers in effectively implementing instruction 
aligned to new math, English language arts, and science standards (e.g., Marrongelle, Sztajn, & 

Smith, 2013), our work has direct implications for how districts and schools might stmcture PD 
better to improve its effectiveness. 

Specifically, our findings suggest implications for how PD is designed and provided as well 
as how its effectiveness is determined. Despite substantial progress in developing a consensus 
around core features of high-quality PD, there has been little call for differentiated PD, or activities 
calibrated to the needs and learning levels of particular teachers. These ideas are not completely new, 
however. Santagata et al. (2012), in discussing variation in teacher response to a PD intervention, 
suggest gauging PD to teachers’ knowledge and needs. In asking what types of content knowledge 
are necessary for effective teaching, Diamond et al. (2014) leave open the idea that such knowledge 
may differ across teachers. Other researchers recommend considering knowledge and experience 
differences among teachers in designing PD (Loucks-Horsley Stiles, Mundry, Love, & Hewson, 
2010) and personalizing PD to address teachers’ particular circumstances and challenges with 
instructional practices (Bitan-Friedlander et al., 2004; Starkey et al., 2009). 

Our findings lend support to the idea of differentiating PD. They show that what teachers 
learn in PD depends on what they already know. This may seem obvious, but current models of 
“one size fits all” PD do not reflect this reality. Our data suggest that teachers can take away very 
different information and strategies from a learning experience, depending on the knowledge they 
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bring to the activity, as well as their own proclivity and assessment of where they would like to 
improve. The PD may not have the intended outcome if teachers have varying responses and 
learning from the PD. Focusing on high-quality PD is not enough; we need to think more about 
how the PD interacts with teachers’ previous knowledge and experiences. 

Our findings suggest how important CK may be for developing deep understanding, echoing 
other research prioritizing the role of CK in teachers’ instructional development (e.g., Cohen & Hill, 
2001; Lee & Luft, 2008; Ma, 1999). But our findings go beyond that to suggest different ways that 
teachers can benefit from high-quality PD and how this may be systematically related to their CK. 
Further, we provide insights into how a PD consisting of a set of principles, activities, and lessons 
can provide some teachers with an opportunity to gain deeper ideas about instmction and ways of 
presenting material while simply providing other teachers with a lesson plan to follow, perhaps with 
“magical instmctional powers” (Cohen, 1990, p. 318) of its own. 

How a teacher reacts plays a large part in determining the extent to which an intervention 
changes what teachers know and how they think. It is not just the intervention itself that matters, 
but the interaction between teachers’ knowledge and their PD experience. While some PD 
interventions have shown reasonable average effects, many more vary in their effects by measurable 
characteristics such as teacher experience (e.g., Borman et al., 2008), and some very intensive PDs 
have had little effect on instmctional practice and student achievement (e.g., Garet et al., 2010). We 
hope that the ideas presented here will play a role in interpreting these studies. 

Designing PD that is calibrated to teachers’ needs and prior knowledge holds great promise 
for increasing the payoff for PD. This could be especially important in light of millions of dollars 
currently going to developing PD to support Common Core State Standards implementation in the 
United States. Further, knowledge of the variation in what teachers learn and do as a result of the 
same PD experiences contributes to our understanding of when and how PD affects teacher 
learning, and could help us become more sophisticated in how we measure and interpret findings of 
PD effectiveness studies. 
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